1. Field of the Invention
This invention relates generally to a system that facilitates the collection, organization, and visualization of user feedback. In one described implementation a system for permitting users of computer software to make feedback as to the content of the software while the software is in use is described.
2. Background of the Related Art
Organizational learning systems are incorporated in such organizations as schools, which teach students a curriculum, and businesses, which teach employees skills. An organizational learning system typically includes at least two basic roles: a learner and a teacher. The learner wishes to learn new concepts, procedures, or other information. The teacher ideally wishes to present the information to the learner in a way that best meets a learner's individual learning needs. However, it often occurs that the learner "stutters" in the learning process or "flow": the learner comes upon a concept, instructions, or information that he or she does not understand, either partially or completely. It is in the teacher's interest to know which information the learner had trouble understanding, if the learner had trouble due to the presentation method or style, and how the presented information can be improved to facilitate learning. Once the teacher has obtained and understood these aspects of the taught information, the teacher can change and improve the presentation of the information to more fully meet the learners' learning needs and facilitate the learning process. A critical resource for the teacher to discover how to improve taught information and the learning organization is learner "feedback", which is the information, comments, suggestions, and questions sent from the learners to the teacher about the taught information.
The most essential feature of any self-regulating, self-correcting, or self-optimizing system is feedback. Without feedback, the system is stagnant: learners or users will continue to have problems learning or interfacing within the system, and have no way to voice those problems to effect improvements. Learners know which parts of taught information gave them the most trouble or could best be improved, and it is essential in the system's implementation to capture and analyze their feedback. It is critical that the learning system gathers feedback and presents the feedback to people who can improve the learning system so that the system may continually improve.
For example, a business organization must continually improve and refine its products and internal processes to remain competitive. A business' ability to optimize its products and services is directly related to how well it can "learn" from its customers' experiences of those products and services through customer feedback. Feedback is also necessary for internal departments and processes within a business. The more effectively that employees can learn a new procedure or suggest improvements to, for example, manufacturing processes, the better will the business perform competitively.
Feedback is just as necessary in education. A student may often reach a point in course material where he or she does not understand the concepts being taught. If the student can indicate to the teacher where the difficult material is, the teacher can be more aware of difficult concepts and can improve the course. Or, the student might eventually struggle to understand a concept and have his or her own suggestions of how to better present the material. If these learning needs of the learner are not addressed using feedback, the learner may become used to skipping over difficult concepts, ignoring their own questions about the concepts since those questions are not answered within the system. If the learner can provide feedback about a course, the course can be improved and the outcome is better learners.
Feedback is also very necessary in other environments besides learning environments. For example, a testing environment in which a product or service, such as software, is being tested for operability, reliability, ease of use, etc., can implement user feedback. Product users and testers can send feedback about products or services which can greatly aid developers in testing and developing those products and services. Similarly, any product, process, service, or issue experienced by users can be improved through the collection and analysis of feedback.
In the past, the need to collect feedback from learners and other types of users has been recognized by organizations. Typically, however, feedback-gathering systems have been inefficient at best. One of the most prevalent problems in prior feedback-gathering systems is motivating users to supply accurate and worthwhile feedback. There are several reasons why users typically resist feedback. One reason is that users complain that giving feedback takes too much time and is too distracting. Conventional feedback-gathering methods typically take the form of supplying the user a questionnaire or similar response opportunity after at least a large portion of the material being taught has been presented or after a service has been experienced. For feedback to be given in any amount of detail, relatively lengthy questionnaires are required which can take a great a deal of time to complete. In addition, since such questionnaires are typically filled out long after any problems were experienced, it may be difficult for the user to remember the details of a problem or suggestion. For example, it may be difficult for a learner to remember all the details of course materials and in which areas he or she had trouble and/or suggestions.
Another reason users typically are not motivated to supply feedback is that they know what is wrong or what needs to be changed, but they don't know enough about the entire situation or concept to explain the problem. That is, a user often does not know or cannot explain the context of the problem or suggestion that is necessary to "place" the feedback in the mind of the recipient. The step of describing the context of a user's problem can therefore take a lot of time and potentially cause misunderstandings between the reviewer or provider and the user.
Other reasons why users are not motivated to supply feedback are that users often do not know whom to give the feedback to and are reluctant to provide critical and useful feedback for fear of possible reprisals. The user may want only certain reviewers to receive his or her feedback, but typically has no control over the visibility of the feedback to reviewers. Also, in many cases, a user may believe that the reviewer does not really want to hear the feedback. A user may believe that nothing will result from their feedback--i.e. that there is little reason to voice their opinion, since they cannot change anything. If users are not motivated to provide feedback, they may start to neglect their own questions and become used to ignoring questions or suggestions they may have about a course subject, product, issue, etc. The user may become accustomed to accepting a product or service with which they are dissatisfied without trying to add their own input about how to improve the product or service.
The prior art feedback systems not only have problems in gathering feedback, they also generally fail to provide an efficient and functional way to organize and present feedback to a teacher, supervisor, or reviewer. Once the gathering of feedback has been accomplished, the reviewer must somehow organize that feedback into logical and relevant categories. This can be quite a problem when a large number of feedback items are collected. Once the feedback is organized, the reviewer must determine a method to present the feedback in a way that will show significant patterns within the feedback and help the reviewer to draw conclusions from the feedback. Since the organization and presentation of feedback can involve substantial amounts of time and effort for the people involved, an organization may have no specific method to view feedback and may just examine a few responses to determine what an "average" user is "complaining about." This method does not consider the individual user and his or her learning needs or other needs. Or, an organization may organize feedback into specific categories, each category having a simple percentage indicating how many learners provided a specific response. Such a system does not allow a reviewer to view feedback in more than a few simple patterns which may not be the most effective way to draw conclusions from feedback. In addition, an individual teacher may want to view a particular subset of the total collected feedback in a way specific to his or her own course material or personal duties. When a system does not provide for presenting feedback to an individual reviewer's needs, the time and effort required to organize and view feedback in relevant ways can in itself be a deterrent to taking steps to improve the learning system.
In view of the described drawbacks of existing feedback systems, what is needed is a feedback system that minimizes the effort and time required of individual users to enter feedback and provides motivation for the users to supply the feedback. The system would provide easy, quick feedback opportunities as the user is learning or using the provided product or service. What is also needed is a feedback system that organizes and presents collected feedback to reviewers quickly and in a form relevant to the reviewer's needs so that the reviewer may easily draw conclusions about aspects of the product or service provided to the user which require improvement.